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ABSTRACT 



This paper presents a model of how multimethod approaches to 
research can be used to understand and measure individual diversity in 
learning, cognitions, and affect. The paper demonstrates the value of 
gathering qualitative data to supplement quantitative findings from 
quasi-experimental research in relation to achievement outcomes, acquiring a 
metacognitive strategy of self -questioning, self-efficacy, and anxiety. An 
aptitude -treatment study was conducted to investigate the comparative effects 
of two instructional approaches on achievement, metacognitive strategy 
acquisition, self-efficacy, and anxiety in a semester-long computer training 
program. There were 22 college students in the intervention group and 23 in 
the comparison group. Simultaneously, continuous qualitative data collection 
and analyses were conducted. Eight case studies were selected for the 
qualitative study. Results indicate complex relationships between initial 
levels of self-efficacy, anxiety, and instructional treatment. The 
simultaneously conducted qualitative studies enabled researchers to situate 
these effects in a holistic context. (Contains 27 references.) (Author/SLD) 
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Overview 

This paper presents a model of how multimethod approaches to research can be used to understand 
and measure individual diversity in learning, cognitions and affect. The paper demonstrates the 
value of gathering qualitative data to supplement quantitative findings from quasi-experimental 
research in relation to achievement outcomes, acquiring a metacognitive strategy of self- 
questioning, self-efficacy, and anxiety. Specifically, an aptitude-treatment-interaction study was 
conducted to investigate the comparative effects of two instructional approaches on achievement, 
metacognitive strategy acquisition, self-efficacy and anxiety in a semester-long computer training 
program. Simultaneously, continuous qualitative data collection and analyses were conducted. 



‘ Full text of intervention study reported in Mclnerney, V., Mclnerney, D. M„ & Marsh, H. W. (1997). Effects of metacognitive 
strategy training within a cooperative group learning context on computer achievement and anxiety: An aptitude-treatment 
interaction study. Journal of Ed ucational Psychology. 89. 4. 686-695. 
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The specific objectives of the research were to: 

1) Compare the achievement, self-efficacy, motivation and anxiety of a sample of students 
undertaking a computing course taught through one of two methods: Direct instruction alone 
or a combination of direct instruction and cooperative, self-regulated learning which 
incorporated training in the metacognitive strategy of self-questioning. 

2) Describe and evaluate each instructional program in operation using qualitative research 
techniques, namely, regular in-depth interviews with a selection of case study students who had 
both high and low anxiety and self-efficacy with regard to computing, analyses of case study 
logbooks and instructor tutorial diary, and molecular analyses of survey data collected at pre- 
and posttest. 

3) Correlate data from case studies regarding their perceptions of the computer skills training they 
received in the context of gaining self-efficacy and self-regulated learning skills, as well as of 
their experiences of anxiety alleviation or exacerbation. 

4) Triangulate the data obtained through quantitative research methods from the full groups 
involved in the quasi-experimental research with those from the qualitative data collected from 
the case studies. 



Method 

Quantitative Study 

The comparison group received a direct instruction (teacher-led) approach (Rosenshine, 1986; 
Rosenshine & Meister, 1995) which has been traditionally used for skills training such as in 
computing. Proponents of this approach maintain that by proceeding in systematic short steps and 
giving guided practice, students' working memory is less overloaded, thereby alleviating anxiety 
(Tobias, 1986). The intervention group received an approach which built on direct instruction by 
adding cooperative self-regulated group work (Johnson & Johnson, 1991; Kagan, 1994; Schunk, 
1990; Zimmerman, 1990; Zimmerman, Bonner & Kovach, 1996) in which students were trained in 
the process of self-questioning following the approach of Alison King (King, 1991a & b, 1992, 
1993, 1994; see also Rosenshine, Meister, & Chapman, 1996). Self-questioning was predicted to 
reduce anxiety and enhance perceptions of control in computing situations, thus increasing self- 
^ efficacy (Bandura, 1988, 1993). 
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An aptitude-treatment-interaction (ATI) design was utilized in the quasi-experimental study 
as it allows for complex analyses of the effects of instructional methods (Cronbach & Snow, 1977; 
Borg & Gall, 1989; Borg, Gall, & Gall, 1993). The ATI study focused on the interaction between 
the two different instructional approaches and the different levels of learners’ self-efficacy, positive 
and negative cognitions, anxiety and prior computing competence. 

Qualitative Study 

The qualitative approaches used in the studies reported are based on the work of Bouma (1996), 
Cohen & Manion (1994), Miles and Huberman (1994), and Wiersma (1991). Among the 
techniques used were: semi-structured, in-depth interviews with case study students held after each 
tutorial over the semester-long computer course; case study logbooks kept throughout the course of 
their training; molecular analyses (Peck & Hughes, 1996) of case study survey data; and the 
instructor’s tutorial diary in which detailed records of weekly tutorials were kept. 

Participants 

Quantitative Study. Two groups of students completing compulsory computer coursework in the 
subject Introduction to Computers were randomly assigned to the two instructional approaches 
taught by the same instructor. There were 22 students in the intervention (metacognitive training 
within a cooperative learning context) group and 23 in the comparison (direct instruction) group. 
The average age of the students was 20 years. 

Qualitative Study . Eight case studies were selected for the qualitative study from the two groups on 
the following basis: The two students (one male and one female) in each of these groups with the 
highest and the lowest anxiety and positive cognitions scores on the Computer Anxiety and 
Learning Measure were invited to participate in the research. 

Measures 



The Computer Competency Checklist (Lawson & Mclnemey, 1994) was used to determine the 
levels of perceived student self-efficacy with regard to specific computing skills to be covered in the 
course (DQS, word-processing, database, and spreadsheet). The Computer Anxiety and Learning 
Measure - CALM (Mclnemey, Mclnemey, & Roche, 1994) was used to determine levels of positive 
and negative cognitions, and computer anxiety. 
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Results 

Significant ATI effects indicated complex relationships between initial levels of self-efficacy, 
anxiety and instructional treatment. The simultaneously conducted qualitative studies enabled the 
researchers to situate these effects (which were based on measures of central tendency) in a holistic 
context where individual parameters of the interactions were the focus of attention. Apparent 
anomalies in the quantitative findings were explicated through the molecular examination of 
individual scores, detailed analyses of the case study interview transcripts and tutorial logbooks, and 
instructor’s tutorial diary. 

Quantitative findings . Those with initially low self-efficacy at the start of the course increased then- 
level of positive cognitions in relation to computing (perceptions of control and a computing self- 
concept were enhanced) after receiving the intervention; those with high initial self-efficacy were 
advantaged by being in the direct instruction group. Students receiving both instructional 
approaches achieved equally well at the end of the course on a range of assessment measures. High 
anxious students receiving the intervention reported less anxiety at the end of their computer 
training course than those in the comparison group except in relation to receiving feedback on 
competency and fear of public embarrassment when using computers. The results of this study are 
reported in detail in Mclnemey, Mclnemey «& Marsh, 1997. 

Qualitative findings. In trying to understand “What is going on here?’’ (Bouma, 1996, p.l69) from 
the perspective of the case studies and the instructor, the following emerged from the qualitative 
data and analyses. 

Metacognitive Strategy Training 

• Metacognitive training in self-questioning enhances self-regulation and learning. 

• Reciprocal questioning develops self-questioning skills and builds positive cognitions. 

• Training in self-questioning develops content knowledge, self-efficacy and self-regulated 
learning skills. 

• Mutual “helping” relieves anxiety and builds confidence. 

• Anxious students can avoid embarrassment through reciprocal peer questioning. 

• Positive cognitions can develop even when anxiety is reported. 

O 
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Cooperative Self-Regulated Groupwork 

• Groupwork helps learning, enhances motivation, builds confidence, and dissipates anxiety. 

• Growing self-efficacy comes from structured groupwork and training in the use of generic 
question stems. 

• Students develop skills in problem-solving 

• Students are more active and involved with cooperative groupwork. 

• Instructor’s role is one of facilitator. 

Direct Instruction 

• Sense of control over learning is absent in teacher-led instruction. 

• Sense of self-efficacy varies considerably according to the tutorial content 

• Reception learning is boring and unmotivating. 

• Anxiety is related to dependence on the instructor in direct instruction. 

• Spontaneous groupwork emerges even within teacher-led instruction. 

• Attendance problems in direct instruction groups are related to lack of student motivation and 
commitment. 

• Instructor’s role is one of expert trainer. 



Conclusion 



Results from the multimethod research techniques used in the present studies provide strong support 
for the educational value of including metacognitive strategy training in self-questioning within a 
cooperative learning context into traditional methods of teaching computing skills to those learners 
who lack confidence and perceptions of control in computing situations, or who fear public 
embarrassment while learning. Learning outcomes in terms of both personal (self-concept, self- 
efficacy and self-regulation) and achievement gains are strongly predicted with such an instructional 
approach. These research findings have considerable implications for the design of instruction in 
general, and for computer training programs in particular. 
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